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HELMET-MOUNTED  CONJUGATE  OPTICAL 
DISPLAY  SYSTEM:  DESIGN  CONSIDERATIONS 

1.0  INTRODUCTION 

Most  terrain  and  objects  viewed  in  real  aircraft  are  located  at  effective  optical  infinity 
(i.e,,  at  greater  than  about  30  feet).  It  has  been  generally  assumed,  therefore,  that  simulator 
imagery  must  also  be  at  effective  optical  infinity  to  produce  a  realistic  visual  simulation.  For 
this  reason,  optical  collimators  are  used  in  many  visual  display  systems.  Among  the 
collimation  methods  used  in  these  systems  are  large  spherical  mirrors,  multiple  element 
windows,  and  various  helmet-mounted,  fiber-optical  systems  (Kelly,  Shenker,  &  Weissman, 
1992).  There  are  several  disadvantages  associated  with  collimated  visual  systems  including 
optical  complexity,  high-cost,  and  large  size.  In  order  to  address  the  problems  of  optical 
complexity  and  high  cost,  a  display  system  has  recently  been  developed  which  uses 
commercial  projectors  and  conventional  rear-projection  screens  to  present  a  simulated  image 
at  about  one  meter  fi’om  the  pilot  (cf,  Thomas  &  Reining,  1990).  This  system,  however,  is 
still  relatively  large  and  may  result  in  certain  perceptual  problems  related  to  the  physical 
proximity  of  the  imagery  to  the  pilot  (Pierce  &  Geri,  1998). 

As  noted  above,  the  visual  display  systems  employed  in  high-fidelity,  full  field  of 
view  flight  simulators  are  typically  either  optically  complex,  or  large,  or  both.  However,  a 
display  system  has  recently  been  described,  which  could  provide  highly  detailed,  collimated 
imagery  using  relatively  simple,  lightweight,  and  inexpensive  optical  components.  The 
system  employs  a  conjugate-optical  projector  and  a  retroreflecting  screen  to  place  separate 
images  at  each  of  the  observer's  eyes  (Fisher,  1992). 

In  addition  to  optical  complexity,  most  helmet-mounted  systems  will  overlay  the 
simulated  image  on  the  instrument  panel  and  elsewhere  in  the  cockpit  where  the  pilot  might 
look.  The  conjugate  optical  display  systems  (CODSs)  can  significantly  reduce  this  problem 
since  the  projected  imagery  is  virtually  undetectable  unless  projected  against  a  retroreflective 
screen.  Also,  an  individual  CODS  has  the  advantage  that  its  imagery  is,  for  the  most  part, 
not  visible  to  observers  other  than  the  user.  Thus,  separate  CODS  could  potentially  be  used 
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to  present  imagery  simulated  from  the  slightly  different  viewpoints,  as  for  example  of  the 
individual  pilots  in  multipilot  aircraft.  On  the  other  hand,  as  with  any  display  device,  there 
are  also  potential  problems  associated  with  the  use  of  a  CODS.  For  instance,  the  CODS  uses 
a  projected  image  that  may  be  distorted  at  large  distances  from  the  center  of  projection. 
Further,  these  distortions  may  be  exacerbated  by  the  off-axis  properties  of  the  retroreflecting 
screen.  Also,  in  a  CODS,  multiple  projection  sources  will  have  to  be  moimted  on  a  single 
helmet  in  order  to  attain  a  wide  field  of  view  (>150°  x  120°).  In  order  to  effectively  use 
multiple  projectors,  each  must  be  of  low  weight,  and  any  associated  optics  must  be  chosen  to 
allow  image  blending  and  to  minimize  obstructing  the  pilots  view  of  both  the  retroreflecting 
screen  and  cockpit  instruments. 

We  discuss  here  some  optical  and  perceptual  issues  that  we  believe  should  be 
considered  in  designing  future  CODSs.  We  will  first  briefly  describe  the  optical  principles 
behind  the  CODS,  and  then  summarize  some  measurements  we  have  made  in  order  to  verify 
the  properties  of  the  projection  optics  and  the  retroreflecting  screen.  We  will  then  discuss 
several  perceptual  issues  related  to  field  of  view,  depth  perception,  and  the  appearance  of 
imagery  displayed  using  a  CODS.  Finally,  we  will  suggest  some  optical  designs  consistent 
with  the  identified  perceptual  limitations. 

2.0  DEFINITION  OF  A  CONJUGATE  OPTICAL  SYSTEM 

2.1  Conjugate  Points  Relative  to  a  Simple  Lens 

Consider  an  optical  element,  such  as  a  glass  lens,  surrounded  by  air.  Consider  also  a 
ray  of  light  that  originates  in  the  air  and  impinges  on  the  lens.  Given  these  conditions,  a 
portion  of  this  ray  will  be  reflected  by  the  glass  lens  surface  back  into  the  air  and  a  portion 
will  be  refracted  into  the  lens.  The  laws  of  reflection  and  refraction  state  that:  (a)  the 
reflected  ray  will  leave  the  lens  surface  at  the  same  angle  at  which  it  impinged  on  the 
surface,  and  (b)  the  refracted  ray  will  enter  the  lens  at  an  angle  that  depends  on  the  properties 
of  the  glass  (or  other  substance)  making  up  the  lens.  The  principle  of  reversibility  states  that 
if  the  direction  of  a  light  ray  is  reversed  after  reflection  or  refraction,  it  will  retrace  its 
original  path.  Further,  if  parallel  light  rays  are  made  to  impinge  on  a  lens,  the  rays  will  be 
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refracted  both  upon  entering  and  leaving  the  lens.  With  a  convex  lens  the  rays  will  converge 
to  a  point,  called  the /oca/  point,  on  the  other  side  of  the  lens.  Similarly,  by  the  principle  of 
reversibihty,  light  diverging  from  the  focal  point  will  be  rendered  parallel  on  the  opposite 
side  of  the  lens. 

Applying  the  above-stated  principles  to  the  optical  system  shown  in  Figure  1,  if  the 
upper  ray  from  the  object  (A)  is  parallel  to  the  lens  axis,  that  ray  will  pass  through  one  focal 
point  (F')  after  being  refracted  by  the  lens.  Further,  if  the  lower  ray  from  point  A  passes 
through  the  other  focal  point  (F),  that  ray  will  exit  the  lens  parallel  to  the  lens  axis  after 
being  refracted.  The  point  of  intersection  (A’)  of  these  two  rays  determines  the  location  of 
the  image  and  its  size.  By  definition,  the  object  and  its  image  are  located  at  conjugate  points 
about  the  lens. 

As  shown  in  Figure  1,  if  an  illimiinated  object  is  placed  on  one  side  of  a  convex  lens, 
and  farther  away  than  the  focal  point  (F),  an  image  of  the  object  will  be  formed  on  the  other 
side  of  the  lens.  Again,  from  the  principle  of  reversibility,  the  object  and  its  image  are 
interchangeable.  That  is,  if  the  object  were  placed  where  its  original  image  was  located,  its 
new  image  would  be  where  the  object  originally  was.  In  fact,  the  word  conjugate  literally 
means  interchangeable.  Note  that  if  the  object  is  moved  closer  to  the  focal  point  on  one  side 


Figure  1.  An  Optical  System  Defining  Conjugate  Points  About  a  Simple  Lens. 
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of  the  lens,  then  the  image  will  move  farther  away  from  the  focal  point  on  the  other  side  of 
the  lens,  and  it  will  be  magnified.  The  opposite  is  the  case  if  the  object  is  moved  farther 
from  the  focal  point.  In  either  case,  the  pair  of  conjugate  points  will  have  changed,  although 
the  new  pair  will  still  be  conjugate. 

2.2  Retroreflecting  Screens 

At  a  typical  reflecting  surface  such  as  a  plane  mirror,  only  light  rays  directed 
perpendicular  to  the  surface  are  reflected  backwards  parallel  to  the  incoming  rays;  all  other 
rays  are  reflected  in  other  directions  (see  Figure  2a).  However,  if  two  reflecting  surfaces  are 
placed  at  right  angles  (see  Figure  2b)  the  incoming  rays  will  be  reflected  twice,  one  by  each 
surface,  such  that  all  rays  will  exit  parallel  to  the  incoming  rays.  Such  an  arrangement  is 
called  a  retroreflector.  This  optical  geometry  is  easily  extended  to  three-dimensional  space 
by  using  three  reflecting  surfaces  that  are  arranged  at  right  angles  like  the  comers  of  a  cube. 
In  fact,  this  type  of  retroreflector  is  called  a  cube  comer  retroreflector. 


a) 


Figure  2.  A  Plane  Mirror  (a),  and  A  Prism  Retroreflector  (b). 

If  numerous,  small  prisms  are  etched  or  otherwise  formed  on  the  surface  of  a  glass  or 
plastic  substrate,  the  surface  will  reflect  light  back  very  nearly  along  its  original  path,  thus 
producing  a  retroreflective  surface.  A  retroreflector  can  also  be  produced  by  embedding 
small  plastic  beads  on  a  surface.  Retroreflecting  screens  are  routinely  used,  for  instance,  in 
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traffic  signs  to  increase  the  amount  of  light  reflected  directly  back  to  a  driver  at  night. 
Retroreflective  material  is  available  in  large  flexible  sheets,  and  it  can  be  moimted  in  many 
configurations.*  We  have  measured  several  of  the  properties  of  retroreflecting  screens, 
which  are  relevant  to  their  use  in  a  helmet-mounted  CODS,  and  those  data  will  be  described 
later  in  this  paper. 

2.3  A  Simple  Conjugate  Optics  Display  System  (CODS) 

Using  the  principles  described  above,  we  can  produce  a  very  simple  optical  system 
that  will  provide  a  conjugate-optical  image.  A  schematic  of  such  a  system  is  shown  in 
Figure  3.  The  liquid  crystal  display  (LCD)  is  analogous  to  point  A  in  Figure  1.  The 
conjugate  optical  system,  which  includes  the  projection  lens,  the  beamsplitter,  the 
retroreflecting  screen,  and  the  optics  of  the  observer’s  eye,  is  analogous  to  the  lens  of  Figure 
1.  The  display  system  of  Figure  3  places  an  image  of  the  LCD  display  on  the  observer's 
retina.  In  terms  of  optical  geometry,  the  LCD  display  and  the  retinal  image  are 
interchangeable  (conjugate),  as  were  points  A  and  A’. 


beamsplitter 

retinal  eye 
image  optics 


retroreflecting 
screen  i 


Figure  3.  A  Simple  Conjugate-Optical  Display  System. 
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3.0  CODS  COMPONENT  EVALUATION 


3.1  Bench  CODS 

In  order  to  investigate  some  of  the  basic  properties  of  the  CODS,  we  designed  and 
built  a  system  that  could  be  mounted  on  an  optical  bench,  and  that  would  allow  us  to  vary 
the  angle  between  the  lines  of  sight  of  the  two  eyes  (i.e.,  the  binocular  vergence  angle). 
Photographs  of  the  bench  CODS  are  shown  in  Figures  4a  and  4b.  The  test  imagery  was 
displayed  on  two,  monochrome  (green)  CRTs.  The  imagery  from  both  CRTs  was  projected 
using  either  a  16  mm  or  25  mm  CCTV  lens  (Edmund  Scientific,  Barrington,  NJ)  and  was 
reflected  by  a  large  beamsplitter.  In  accordance  with  the  principles  described  earlier  (see 
Figure  3),  the  beamsplitter  of  the  CODS  directed  a  portion  of  the  light  to  a  retroreflecting 
screen  that  reflected  the  light  back  through  the  beamsplitter  to  the  observer's  eyes.  The 
CRTs  and  associated  electronics  were  designed  to  produce  1,000  lines  independently  of 
each  other,  but  for  the  present  system  evaluation,  both  CRTs  displayed  the  same  imagery, 
which  was  provided  by  a  standard  VHS  video  cassette  recorder. 

The  level  of  ocular  vergence  can  be  specified  as  the  angle  formed  by  the  lines  of 
sight  of  the  two  eyes.  Thus,  when  viewing  (i.e.,  fixating  upon)  a  distant  object  with  both 
eyes,  the  vergence  angle  is  small,  but  it  increases  as  the  distance  to  the  viewed  object 
decreases.  Vergence  could  be  changed  in  our  bench  CODS  by  varying  the  angle  between  the 
mounted  CRTs  (see  Figure  4b).  The  retroreflecting  screen  was  mounted  such  that  both  its 
angle  relative  to  the  observer's  line  of  sight  and  its  distances  from  the  observer  could  be 
varied  independently.  Using  the  25  mm  projection  lens  and  a  retroreflecting  screen  distance 
of  36",  the  1"  diameter  CRT  provided  an  image  over  a  field  of  view  of  about  31°  x  24°. 


6 


(b) 


Vergence 

Adjustment 


CRTs 

Projection 

Lens 


Beamsplitter 


Figure  4.  Bench  CODS,  (a)  Full  View;  (b)  Close-up  of  Optics. 
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3.2  Measurement  of  Retroreflecting  Screen  Properties 


The  directional  properties  of  a  retroreflecting  screen  can  be  characterized  in  two 
fundamentally  different  ways.  Following  the  nomenclature  used  by  one  manufacturer  ,  the 
angular  response  of  a  retroreflecting  screen  can  be  measured  by  aligning  the  source  and 
detector  and  then  changing  the  entrance  angle,  P,  they  make  with  the  normal  to  the  surface 
of  the  screen  (see  Figure  5,  top).  Another  property,  reflective  efficiency,  provides  a  more 
direct  assessment  of  the  retroreflecting  properties  of  the  screen,  and  can  be  measured  by 
directing  the  source  along  a  normal  to  the  surface,  and  then  varying  the  observation  angle,  a, 
of  the  detector  relative  to  the  normal  (see  Figure  5,  bottom). 


Figure  5.  Configurations  for  Measuring  the  Directional  Properties  of 
Retroreflecting  Screens,  a  =  observation  angle,  fl  =  entrance  angle. 
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In  the  present  study,  angular  response  was  measured  by  keeping  the  source  and 
detector  stationary  and  rotating  the  retroreflecting  screen.  The  distance  from  the  source  to 
the  screen  was  also  varied.  A  PR-1500  Photometer  (Photo  Research  Inc.,  Chatsworth,  CA) 
was  used  for  all  measurements  and  was  placed  at  the  eye  point.  We  did  not  measme 
reflective  efficiency,  although  we  have  replotted,  in  polar  coordinates,  one  manufacturer's 
data^  (see  Sect  5.5,  p.  21). 

3.2.1  Angular  Response  Curves 

Angular  response  curves  for  the  high-gain  screen  used  in  the  bench  CODS  are  shown 
in  Figure  6.  These  cxuves  show  measured  luminance  as  a  fimction  of  the  entrance  angle  P, 
between  the  detector/source  and  the  normal  to  the  screen  (see  also  Figme  5,  top).  The  data 
verify  the  well-documented  properties  of  the  retroreflecting  screen  for  several  viewing 
distances  from  1.0  ri  to  3.0  m.  At  a  given  viewing  distance,  image  luminance  increases 
gradually  with  an  increase  in  screen  angle  from  O'*  to  about  25°.  Image  luminance  then 
decreases  for  larger  viewing  angles,  effectively  asymptoting  at  about  60°.  Although  there  is 
a  significant  decrease  in  image  luminance  with  viewing  distance,  the  general  form  of  the 
function  relating  image  luminance  to  viewing  angle  is  the  same  for  the  various  viewing 
distances  tested.  Thus,  the  decrease  in  image  luminance  with  viewing  distance,  which  is 
evident  in  the  data  of  Figure  6,  does  not  interact  with  the  viewing  angle. 

In  obtaining  the  measurements  shown  in  Figure  6,  we  varied  the  observation  angle 
using  the  source  and  detector  configmation  shown  in  Figure  5  (top).  It  would  also  be 
possible  to  use  the  component  configuration  of  Figure  5  (bottom),  and  in  addition  change  the 
entrance  angle.  The  resulting  data  would  then  be  determined  by  both  of  the  retroreflecting 
screen  properties  described  above.  Specifically,  data  obtained  with  the  source  directed 
normal  to  the  screen  and  the  detector  directed  at  various  locations  across  the  screen,  at  least 
along  the  horizontal  meridian,  would  probably  provide  important  information  especially  in 
characterizing  wide-field  CODSs.  Data  of  this  kind  would  also  help  to  determine  the  extent 
to  which  angular  response,  reflective  efficiency,  and  other  measures  of  directionality  may  be 
a  property  of  the  CODS  as  a  whole,  as  opposed  to  just  the  retroreflecting  screen. 
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Figure  6.  Angular  Response  Functions  of  a  Retroreflecting  Screen. 

4.0  PERCEPTUAL  EFFECTS  RELATED  TO  CODS  IMAGEkY 
4.1  Apparent  Depth  of  CODS  Imagery 

One  of  the  potential  advantages  of  a  CODS  as  compared  to  real-image  or  collimated- 
image  displays  is  that  both  the  apparent  distance  of  a  viewed  image  and  the  relative  depth  of 
objects  (i.e.,  within  the  image  at  a  given  apparent  distance)  can  be  varied.  The  apparent 
distance  can  be  altered  by  changing  the  vergence  of  the  CODS  (i.e.,  the  relative  angle  of  the 
left-  and  right-eye  channels).  The  relative  depth  of  objects,  however,  is  a  stereoscopic 
phenomenon,  and  producing  it  would  require  presenting  different  views  of  the  image  to  each 
eye.  Both  types  of  perceived  depth  will  be  discussed  below. 

It  is  perhaps  not  surprising  that  if  the  right-  and  left-eye  channels  are  set  such  that 
their  respective  images  converge  on  the  retroreflecting  screen,  then  the  projected  image 
appears  to  the  user  to  be  located  at  the  plane  of  the  retroreflecting  screen.  If  eye  vergence  is 
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then  reduced  without  changing  the  screen  distance,  the  image  appears  behind  the  screen.  In 
this  latter  case,  if  the  edges  of  the  screen  are  visible,  the  perceptual  effect  is  one  of  viewing 
imagery  through  a  window  or  aperture.  This  is  true  even  for  no  convergence  at  the  farthest 
screen  distance  that  we  used,  which  was  about  20  ft.  At  distances  greater  than  about  15ft, 
the  apparent  separation  between  the  screen  "window"  and  the  image  plane  is  small  and 
probably  of  no  practical  importance.  However,  the  separation  referred  to  above  may  be 
perceptually  more  salient  for  screens  significantly  larger  than  the  one  we  used  (about  3  ft  x  2 
ft). 

4.2  Cue  Conflict  in  Image  Segmentation 

The  visual  impression  of  depth  is  largely  a  consequence  of  the  slightly  different 
views  of  the  world  available  to  the  two  eyes.  The  slight  difference  between  views  is  called 
binocular  disparity,  and  the  resulting  perception  of  depth  is  called  stereopsis.  The  visual 
system  uses  various  cues  to  segment  a  visual  scene  into  objects  and  surfaces.  Two  of  the 
more  salient  segmentation  cues  are  discontinuities  in  stereoscopic  depth  and  discontinuities 
in  surface  properties,  such  as  luminance,  color,  line  patterns,  and  texture.  Under  most 
viewing  conditions,  objects  at  different  depths  also  have  different  surface  properties,  and  so 
the  two  cues  are  consistent.  Under  some  viewing  conditions,  however,  there  may  be 
differences  in  one  of  these  properties  hut  not  the  other,  and  in  this  case  a  cue  conflict  will 
occur.  This  type 

surface  occludes  a  more  distant  surface,  portions  of  the  farther  surface  may  not  be 
visible  to  one  or  both  eyes.  Again,  these  unpaired  regions  arise  because  each  of  the  eyes  has 
a  slightly  different  view  of  the  surfaces,  and  as  a  result  each  eye  sees  the  surfaces  occluded 
to  different  extents.  The  portion  of  a  surface  visible  to  only  one  eye  is  said  to  be  half- 
occluded. 

Researchers  in  the  area  of  binocular  vision  often  produce  the  perception  of 
stereoscopic  depth  by  using  devices  which  present  slightly  different  views  of  the  same  scene 
to  the  two  eyes.  These  devices  are  called  stereoscopes  and  are  similar,  in  principle,  to  the 
3D-viewers  available  commercially.  A  potential  problem  in  using  stereoscopes  is  that  half 
occlusions  can  be  produced  which  are  not  consistent  with  those  that  occur  when  viewing  the 
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same  visual  scene  with  two  eyes  in  the  real  world.  Half-occlusions  of  this  kind  are  called 
ecologically  invalid,  whereas  those  that  are  consistent  with  vision  in  the  real  world  are  called 
ecologically  valid.  Shimojo  and  Nakayama  (1990)  have  shown  that  under  ecologically 
valid,  half-occlusion  conditions  the  unpaired  portions  of  the  viewed  surfaces  are  always 
bound  in  depth  to  the  more  distant  surface.  By  contrast,  imder  ecologically  invalid 
conditions  the  unpaired  regions  are  seen  alternately  by  the  two  eyes  (a  condition  known  as 
binocular  rivalry)  and  appear  to  have  ambiguous  depth. 

Examples  of  occlusion  configurations  that  might  be  found  imder  natural  viewing 
conditions,  and  hence  are  ecologically  valid,  are  shown  in  Figure  7.  Shown  at  the  top  of  the 
figure  are  the  two  eyes  (left  eye  [LE]  and  right  eye  [RE])  of  an  observer  who  is  viewing  two 
surfaces-one  surface  is  larger  and  farther  fi-om  the  observer  than  the  other  surface.  Because 
the  smaller  surface  occludes  a  restricted  portion  of  the  larger  surface,  this  configuration  is 
referred  to  as  a  "spot  occluder."  As  can  be  seen  fi-om  the  sighting  lines  drawn  in  the  figure, 
in  this  configuration  two  small  portions  (labeled  "LE  only"  and  "RE  only")  of  the  farther 
surface  are  each  visible  to  only  one  of  the  two  eyes.  In  addition,  these  two  small  portions  of 
the  farther  surface  are  in  the  temporal  visual  field  of  the  eye  to  which  they  are  visible. 

Another  ecologically  valid  configuration  is  shown  in  the  bottom  diagram  in  Figure  7. 
For  obvious  reasons,  this  configuration  is  called  a  "window  occluder."  Again,  there  are  two 
portions  of  the  farther  surface  that  are  each  visible  only  to  one  eye.  In  this  configuration, 
however,  the  two  portions  are  in  the  nasal  visual  field  of  the  eye  to  which  they  are  visible. 

In  order  to  be  seen  in  stereoscopic  depth,  an  object  or  surface  must  present  slightly 
different  views  to  the  two  eyes.  However,  as  noted  above,  for  both  the  spot  occluder  and  the 
window  occluder,  there  are  portions  of  the  image  which  are  seen  only  by  one  eye,  and  which 
therefore  cannot  imambiguously  be  associated  with  either  the  nearer  or  farther  surface. 
Despite  this  ambiguity,  the  half  occluded  regions  described  above  are  correctly  seen  as  part 
of  the  farther  surface.  It  has  been  suggested  (Nakayama  &  Shimojo,  1990)  that  the  resulting 
percept  is  related  to  the  fact  that  the  spot  occluder  and  the  window  occluder  are  naturally 
occurring  configurations  that  stimulate  the  two  eyes  in  a  way  that  humans  have  evolved  to 
interpret  veridically. 
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As  is  the  case  with  a  more  conventional  stereoscope,  both  ecologically  valid  and 
ecologically  invalid  occlusion  configurations  may  be  created  using  a  CODS.  The  top 
diagram  in  Figure  8  shows  a  viewing  situation  that  could  result  if  the  vergence  angle  of  the 
two  channels  of  the  CODS  is  set  so  as  to  place  the  fixation  plane  nearer  to  the  subject  than 
the  retroreflecting  screen.  If  unpaired  imagery  is  projected  off  the  sides  of  the  retroreflective 
panel,  ecologically  invalid  half-occlusions  will  be  produced.  This  near-vergence  condition 
produces  half-occlusions  similar  to  those  of  a  “spot  occluder”,  which  are  in  themselves 
perceptually  valid.  The  problem  arises  because  the  binocularly  unpaired  projected  imagery 
is,  according  to  image  surface  properties,  supposed  to  perceptually  bind  with  the  closer 
surface  (see  speckled  texture  areas  in  Figure  8).  However,  according  to  the  stereoscopic 
half-occlusion  cue,  these  same  regions  are  supposed  to  bind  to  the  farther  surface  (see 
marbled  texture  areas  in  Figure  8).  Consequently,  the  image  creates  a  cue  conflict,  and 
hence  ambiguous  depth  cues  and  disconcerting  visual  imagery.  By  comparison,  the  viewing 
situation  that  results  if  the  viewing  angle  of  the  CODS  is  set  so  as  to  place  the  fixation  plane 
farther  fi-om  the  subject  than  the  retroreflecting  screen  is  shown  in  the  lower  diagram  of 
Figure  8.  In  this  case,  rmambiguous  cues  are  provided  by  both  the  surfaces  properties 
(texture  in  this  case),  and  the  half-occlusions. 
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Figure  7.  Half-Occlusion  Viewing  Conditions. 
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4.2.1  Recommendations 


Based  on  these  observations,  we  recommend  that  the  CODS  vergence  angle  be  set  so 
that  the  plane  of  fixation  is  at  a  distance  greater  than  or  equal  to  the  distance  of  the 
retroreflecting  screen  (see  Figure  8,  bottom).  As  mentioned  above,  this  configuration  will 
give  the  impression  of  looking  through  a  window,  which  is  particularly  appropriate  for  flight 
simulator  applications.  On  the  other  hand,  there  are  some  situations,  for  instance  when  the 
system  is  used  to  display  virtual  instrumentation,  in  which  it  may  be  desirable  to  have  the 
screen  farther  away  than  the  fixation  plan.  In  these  situations,  all  of  the  nearer  imagery  must 
be  reflected  to  both  eyes  (i.e.,  the  screen  must  be  larger  than  the  visible  projected  imagery). 
If  it  is  not,  the  half-occlusion  imagery  will  create  uncomfortable  and  unrealistic  viewing 
conditions. 

5.0  CODS  OPTICAL  DESIGN  CONSIDERATIONS 

5.1  Image  Tiling  to  Increase  Spatial  Detail 

In  a  helmet-mounted  version  of  the  CODS  (Fisher,  1992),  the  same  visual  scene  is 
displayed  on  two  CRTs  and  is  reflected  and  viewed  through  individual  planar  beamsplitters 
situated  in  firont  of  each  eye.  Given  that  display  devices  currently  available  for  helmet- 
mounted  applications  have  relatively  low  pixel  counts  (i.e.,  at  most  1280  x  1024),  and  given 
that  at  reasonable  screen  distances  (see  below)  the  imagery  projected  from  a  single  source 
will  in  general  not  subtend  more  than  about  45°,  image  detail  high  enough  to  support  most 
flight  simulator  applications  would  require  multiple  displays,  each  subserving  a  portion  of 
the  required  field  of  view.  To  obtain  a  larger  field  of  view  additional  projection  sources  must 
be  added.  We  will  discuss  next  some  of  the  issues  that  arise  in  using  multiple  projection 
sources  in  a  CODS. 
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Figure  8.  Cue-Conflict  and  Half-Occlusion 
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A  conceptually  simple  method  of  optically  tiling^  multiple  image  sources  in  a  CODS, 
would  be  to  use  two  or  more  planar  beamsplitters.  A  diagram  of  such  a  piecewise  planar 
beamsplitter  is  shown  in  Figure  9.  In  the  context  of  the  CODS  shown  in  Figure  3,  one  of 
the  beamsplitters  of  Figure  9  might  be  positioned  directly  in  front  of  one  of  the  eyes,  with 
the  other  beamsplitter  located  in  the  temporal  visual  field  and  thus  presenting  peripheral 
imagery.  The  most  obvious  problem  associated  with  this  beamsplitter  configuration  is  the 
unavoidable  seam  between  each  adjacent  pair  of  beamsplitters.  It  is  expected  that  the 
visibility  of  these  beamsplitter  seams  will  be  relatively  low  because  they  are  located  much 
closer  to  the  eye  than  the  imagery  being  viewed  by  the  user,  which  will  serve  to  blur  the 
edges  of  the  seam.  Also,  the  seam  will  usually  be  located  in  the  visual  periphery  (typically 
beyond  25®),  because  the  CODS  beamsplitters  move  with  the  head,  and  so  the  user's  gaze 
will  most  often  be  directed  through  the  center  of  the  front  beamsplitter  even  when  imagery 
reflected  from  side  screens  is  being  viewed. 


Figure  9.  Piecewise-Planar  Beamsplitter. 

The  problem  of  beamsplitter  seams  can  also  be  avoided  by  using  a  curved  version  of 
the  45°  planar  beamsplitter,  which  would  appear  as  a  section  of  a  cone  with  its  pointed  end 
removed  (see  Figure  10).  A  cone  with  its  end  removed  is  called  a  One  major 

consideration  with  using  a  finstrum  in  this  application,  however,  is  that  the  imagery  is 
reflected  off  of  the  curved  surface,  which  causes  the  image  to  diverge,  much  like  a  convex 
(e.g.,  fun  house)  mirror  does.  As  a  result  of  the  principle  of  optical  reversibility,  the 
distortion  is  reversed  (and  hence  removed)  in  the  portion  of  the  image  returning  from  the 
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screen,  which  is  reflected  again  at  the  curved  surface  of  the  beamsplitter.  However,  the 
image  viewed  by  the  user  remains  distorted  because  it  is  transmitted  rather  than  reflected  at 
the  curved  surface.  In  order  to  remove  the  distortion  in  the  viewed  image,  the  inverse  of  the 
beamsplitter  distortion  function  would  have  to  be  introduced  into  the  generated  image.  This 
can  be  done  relatively  easily.  Nevertheless,  it  is  one  more  step  that  will  have  to  be  added  to 
the  image  generation  process,  and  separate  calculations  may  have  to  be  done  for  each 
beamsplitter. 

We  did  not  notice  significant  distortions  with  the  planar  beamsplitter  used  in  either 
of  our  devices,  although  such  distortions  might  be  more  obvious  when  higher  resolution 
imagery  is  used.  Again,  however,  given  a  head-tracked  system,  displayed  imagery  will  be 
viewed  primarily  through  the  center  of  the  front  beamsplitter  where  distortion  will  be 
minimal.  Fiirther,  any  distortions  of  this  kind  in  the  imagery  presented  through  the 
peripheral  planar  beamsplitters  will  probably  not  be  visible.  Further  engineering  analysis  of 
the  feasibility  of  the  frustrum  beamsplitter  is  recommended. 


Figure  10.  Frustrum  Beamsplitter. 


Visible  seams  are  a  problem  associated  with  many  rear-projection  display  screens. 
The  CODS,  of  course,  might  also  have  seams  between  adjacent  retroreflecting  screens, 
although  these  will  be  much  less  conspicuous  for  two  reasons.  First,  the  retroreflecting 
material  is  much  thinner  and  lighter,  and  more  easily  mounted  than  most  projection  screen 
materials.  This  allows  the  seams  to  be  narrower,  and  makes  it  unnecessary  to  contend  with 
obtrusive  moimting  hardware.  In  addition,  in  a  CODS,  the  imagery  on  both  sides  of  the 
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seam  will  have  originated  from  a  single  projector,  thus  reducing  the  differences  in  the  image 
across  the  seam. 

5.2  High-Resolution  Image  Source 

As  noted  earlier,  a  25  mm  projection  lens,  a  screen  distance  of  36",  and  a  1"  diameter 
CRT  provided  a  31°  x  24°  image  in  our  CODS  system.  Clearly,  several  optical  channels 
will  he  required  to  produce  a  high-resolution,  wide-field  CODS.  Since  multiple  displays 
will  be  required  and  all  image  sources  will  have  to  be  mounted  on  the  same  hehnet,  it  is 
imperative  that  the  display  device  be  a  microdisplay  that  is  lightweight,  bright,  full  color, 
and  provide  at  least  a  IK  x  IK  image.  Microdisplay  technology  is  evolving  rapidly  and  a 
solution  suitable  for  a  CODS  maybe  available  in  the  next  year  or  two. 

Many  other  technologies  have  been  suggested  for  use  as  image  somces  in  helmet- 
mounted  displays,  but  it  appears  that  none  is  currently  capable  of  providing  the  required 
small  pixel  size  in  a  sufficiently  small  package  (Lowe,  1997).  Another  possibility  has 
recently  surfaced  in  the  form  of  laser  projection  systems,  but  these  are  only  in  the  very  early 
stages  of  development  (Glenn,  Holton,  &  Dixon,  1997).  Laser  projectors  are  capable  of 
small  pixel  sizes,  but  the  optics  and  circuitry  required  to  produce  a  usable  raster  are 
relatively  complex.  In  short,  as  was  the  case  when  the  first  CODS  was  first  proposed,  the 
image  source  is  the  major  obstacle  to  implementing  this  system. 

5.3  The  Retroreflecting  Screen 

As  noted  earlier,  the  data  of  Figure  6  were  obtained  using  a  relatively  high  gain 
screen,  which  means  that  the  image  luminance  was  selectively  increased  near  the  center  of 
the  screen  at  the  expense  of  luminance  at  the  edges  of  the  screen.  This  function  would  be 
different  for  different  screen  gains.  In  addition,  it  should  be  noted  that  many  types  of 
retroreflective  screens  are  flexible  and  can  easily  be  applied  to  a  curved  surface.  Suitably 
curving  the  retroreflective  surface  can  also  effectively  change  this  function  by  reducing  the 
variations  in  entrance  angle,  p.  Of  course,  as  is  the  case  with  all  display  systems,  the  screen 
chosen  for  a  particular  CODS  S5^tem  will  be  determined  by  the  application. 
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It  should  be  noted  that  significant  specular  reflection  may  occur  when  the  projected 
image  is  close  to  normal  to  the  retroreflecting  screen.  This  specular  reflection  is  caused  by 
the  surface  material  of  the  screen,  and  is  not  related  to  the  retroreflection  from  the  functional 
screen  structure  (glass  beads,  ridges,  etc.).  Our  preliminary  observations  suggest  that  an 
incidence  angle  of  about  3°  or  4°  from  normal  to  the  surface  is  sufficient  to  reduce  specular 
reflection  to  acceptable  levels.  It  is,  in  principle,  possible  to  angle  all  screens  in  a  CODS 
such  that  no  portion  of  any  screen  is  perpendicular  to  the  user's  line  of  sight.  As  a  practical 
matter  this  will  be  easier  to  accomplish  when  simulating  imagery  for  aircraft  with  smaller 
windows,  each  covering  a  relatively  small  field  of  view  (e.g.,  a  C-130).  For  aircraft 
simulations  providing  a  wide  field  of  view,  the  angle  of  the  screens  to  the  user's  line  of  sight 
may  have  to  be  large,  which  may  introduce  other  optical  or  perceptual  problems  related  to 
the  resulting  differences  in  viewing  distances  to  various  portions  of  the  screen. 

5.4  Optical  Quality  Across  the  Field  of  View 

The  bench  CODS  described  earlier  (see  Section  3.1)  was  constructed  using 
inexpensive  components  which  were  not  necessarily  well-matched  optically.  In  particular, 
the  projection  lenses  were  designed  for  use  with  video  cameras  and,  therefore,  did  not 
accurately  image  the  curved  screen  of  the  CRTs.  This  component  mismatch  resulted  in  a 
slight  difference  in  focus  across  the  displayed  image,  which  we  presume  to  be  related  to  a 
slight  curvature  in  the  projected  image  plane.  If  a  flat  retroreflecting  screen  intercepts  this 
curved  plane  at  a  tangent,  then  the  imagery  will  appear  to  have  best  focus  at  the  center  and 
progressively  worse  focus  in  the  periphery.  We  also  noticed  that  imagery  could  be  produced 
with  poor  focus  at  the  center,  better  focus  in  an  intermediate  annular  region,  and  poor  focus 
again  in  the  periphery.  We  attribute  this  latter  condition  to  the  screen  intercepting  the 
concave  image  plane  at  a  point  nearer  to  the  observer  than  the  tangent  to  that  plane.  It 
should  be  noted  that  the  portion  of  the  image  that  is  out  of  focus  will,  in  a  head-tracked 
system,  remain  in  the  visual  periphery  where  high  detail  cannot  be  seen  in  any  case.  This 
possibility  could  be  investigated  by  checking  to  see  if  field  of  view  changes  at  different 
viewing  distances.  Also,  anyone  designing  a  CODS  might  consider  further  investigating  the 


20 


effects  on  image  homogeneity  of  both  viewing  distance,  and  of  using  less  than  fully 
overlapped  binocular  imagery. 

5.5  Channel  Crosstalk  and  Cyclopean  Viewing 

As  noted  earlier,  the  directional  viewing  properties  of  the  CODS  are  largely  a 
function  of  the  reflective  efficiency  of  the  retroreflecting  screen.  Although  we  have 
described  here  only  binocular  CODSs,  it  is  also  possible  to  construct  a  cyclopean  system 
wherein  the  imagery  supplied  by  a  single  source  is  viewed  by  both  eyes  after  reflection  by 
the  retroreflecting  screen.  Although  this  results  in  a  simpler  system,  it  does  not  fully  exploit 
the  potential  of  a  CODS  for  presenting  wide-field,  stereoscopic  imagery.  Another  limitation 
of  the  cyclopean  approach  is  that,  as  a  consequence  of  the  reflective  efficiency  of  the 
retroreflecting  screen  (see.  Section  3.2),  the  imagery  to  each  eye  will  be  significantly 
reduced.  The  relative  luminance  under  binocular  and  cyclopean  viewing  configurations  is 
shown  in  Figure  11,  where  we  have  plotted  in  polar  coordinates  one  manufacturers  reflective 
efficiency  data^  for  the  retroreflective  screen  used  in  our  CODS.  The  reflective  efficiency 
plot  shown  in  Figure  1 1  assumes  a  200  cm  eye-screen  distance  and  a  6  cm  interpupillary 
distance  (IPD).  The  angular  coordinates  in  the  plot  have  been  expanded  by  a  factor  of  60  so 
that  the  effects  of  small  angle  changes  can  be  better  appreciated. 

hi  Figure  11,  the  projection  and  viewing  angles  associated  with  binocular  viewing 
are  indicated  by  the  two  solid  double  arrows.  The  double  arrows  labeled  Bl,  and  Br  show 
the  angular  location  of  the  source  and  receiver  (i.e.,  eye)  for  the  left-  and  right-image 
channels,  respectively.  Note  that  in  this  configuration  the  reflective  efficiency  is  maximal. 
However,  there  is  a  potential  problem  with  binocular  systems  in  that  the  imagery  associated 
with  one  eye  may  be  visible  to  the  other  eye.  This  phenomenon  is  known  as  crosstalk,  and  it 
results  in  ftie  appearance  of  "ghost"  images.  As  can  be  seen  fi'om  Figure  11,  however,  the 
amount  of  light  directed  to  one  eye  and  seen  by  the  other  is  relatively  small,  although  it  may 
vary  substantially  with  screen  distance  and  EPD. 

The  projection  and  viewing  angles  associated  with  cyclopean  viewing  are  indicated 
by  the  dashed  arrows,  labeled  C-source,  Ct-receiver,  and  CR-receiver.  In  this  case,  the 
source  is  located  between  the  eyes  and  the  result  is  a  significant  reduction  in  image 
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luminance  to  the  two  eyes.  This  is  because  the  reflective  efficiency  function  is  such  that  in  a 
Cyclopean  system  the  Imninance  fall-off  is  very  steep  at  the  observation  angle  between  the 
source  and  receiver.  Furthermore,  in  the  cyclopean  system,  slight  changes  in  the  observation 
angle,  as  may  result,  for  instance,  from  helmet  slippage,  will  cause  dramatic  changes  in  the 
relative  luminance  levels  to  the  two  eyes.  Moreover,  the  change  in  luminance  will  be  in 
opposite  directions,  so  the  image  to  one  eye  will  get  much  brighter  while  that  to  the  other 
will  get  much  dimmer.  Given  the  inefBciency  and  instability  of  the  cyclopean  images  to  the 
two  eyes  we  would  not  recommend  this  configuration  for  use  in  a  CODS. 

Finally,  the  data  of  Figure  11  indicate  that  reflective  efficiency  also  has 
consequences  in  binocular  CODSs.  When  two  image  projectors  are  used,  one  for  each  eye, 
there  can  be  significant  crosstalk  between  the  signals  to  the  two  eyes.  Again,  the  application 
will  determine  whether  this  is  a  significant  problem. 


Luminance  Ql  receiver 


Figure  11.  Cyclopean  vs.  Binocular  Source-Receiver  Relative  Efficiency. 
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7.0  NOTES 


1.  One  manufacturer  maybe  contacted  at:  Screen  Products,  3M  Center  220-7W-06,  St.  Paul, 
MN  55144.  Phone:  612-733-4403. 

2.  The  term  "1,000  lines"  here  refers  only  to  the  video  signal  accepted  by  the  display 
electronics.  We  have  not  measured  the  spatial  resolution  of  this  CRT,  but  it  is  imlikely 
to  be  1,000  lines.  Further,  this  is  a  monochrome  system,  whereas  a  high-fidelity  CODS 
will  require  full  color.  See  Section  5.2  for  a  more  complete  discussion  of  display  devices 
that  may  potentially  be  suitable  for  use  in  a  CODS. 

3.  See,  2M  Special  Effects  Projection  Screens,  3M  Industrial  Optics  Product  Bulletin,  98- 
0439-4177-6(18.25)Rl. 

4.  Image  tiling  as  described  here  should  not  be  confused  with  image  blending,  which  is 
typically  performed  by  the  image  generator  for  the  purpose  of  minimizing  the  borders 
between  adjacent  images  displayed  using  multiple  projectors.  In  addition  to  providing  a 
larger  field  of  view,  multiple  projectors  can  also  provide  high-resolution  insets 
composed  of  smaller  but  more  highly  detailed  images,  hi  order  to  be  useful,  these  insets 
must  be  repositioned  as  the  user's  head  (and  preferably  also  eyes)  moves,  in  order  that 
they  remain  aligned  with  the  high-acuity  portion  of  the  user's  visual  system.  The  CODS 
design  in  no  way  precludes  image  blending,  which  would  in  fact  be  particularly 
efficacious  in  this  case  sinee  head  tracking  would  already  be  implemented,  thus  assuring 
that  the  blended  regions  are  kept  in  the  visual  periphery,  and  hence  relatively 
inconspicuous. 
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